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REMARKS 

The present application was filed on February 6, 2001 with claims 1-38. In the outstanding 
Office Action, the Examiner: (i) rejected claims 1-38 under 35 U.S.C. § 1 03(a) as being unpatentable 
over U.S. Patent No. 6,73 1 ,307 to Strubbe et al. (hereinafter "Strubbe") in view of U.S. Patent No. 
6,751,661 to Geddes (hereinafter "Geddes"); and (ii) rejected claims 28 and 35 under 35 U.S.C. 
§ 103(a) as being unpatentable over Strubbe in view of Geddes in further view U.S. Patent No. 
6,675,356 to Adler (hereinafter "Adler"). 

In this response, Applicants: (i) cancel without prejudice claims 2, 3, 1 1, 12, 20 and 21; and 
(ii) amend the pending independent claims. 

Regarding the pending independent claims, while Applicants believe that such claims, as 
pending prior to this present Amendment, are patentably distinct over Strubbe, Geddes and Adler, 
alone and in combination, Applicants have nonetheless amended the pending independent claims 
in a sincere effort to expedite the present application through to allowance. 

More particularly, the pending independent claims have been amended to recite that "intent 
determination comprises resolving referential ambiguity associated with the one or more users and 
the one or more devices in the environment based on at least a portion of the received multi-modal 
data." This is the concept of referential ambiguity resolution. 

As illustratively explained at page 7, line 7, through page 8, line 18, of the present 
specification: 

The vehicle application lends itself also to an understanding of the concept of 
referential ambiguity resolution . Consider that there are multiple users in the vehicle and that 
the multi-modal conversational computing system 10 is coupled to several devices (e.g., 
telephone, radio, television, lights) which may be controlled by user input commands 
received and processed by the system. In such a situation, not only is there multi-modal 
input, but there may be multi-modal input from multiple occupants of the vehicle. 

Thus, the system 10 must be able to perform user reference resolution, e.g., the 
system may receive the spoken utterance, "call my office," but unless the system can resolve 
which occupant made this statement, it will not know which office phone number to direct 
an associated cellular telephone to call. The system 10 therefore performs referential 
ambiguity resolution with respect to multiple users by taking both audio input data and image 
data input and processing it to make a user resolution determination. This may include 
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detecting speech activity and/or the identity of the user based on both audio and image cues. 
Techniques for accomplishing this will be explained below. 

Similarly, a user may say to the system, "turn that off/' but without device reference 
resolution, the system would not know which associated device to direct to be turned off. 
The system 10 therefore performs referential ambiguity resolution with respect to multiple 
devices by taking both audio input data and image data input and processing it to make a 
device resolution determination. This may include detecting the speaker's head pose using 
gross spatial resolution of the direction being addressed, or body pose (e.g., pointing). This 
may also include disambiguating an I/O (input/output) event generated previously and stored 
in a context manager/history stack (e.g., if a beeper rang and the user asked "turn it off," the 
term "it" can be disambiguated). Techniques for accomplishing this will be explained below. 

In addition, the system 10 may make a determination of a vehicle occupant's mood 
or emotional state in order to effect control of other associated devices that may then effect 
that state. For instance, if the system detects that the user is warm or cold, the system may 
cause the temperature to be adjusted for each passenger. If the passenger is tired, the system 
may cause the adjustment of the seat, increase the music volume, etc. Also, as another 
example (not necessarily an in- vehicle system), an application interface responsiveness may 
be tuned to the mood of the user. For instance, if the user seems confused, help may be 
provided by the system. Further, if the user seems upset, faster executions are attempted. 
Still further, if the user is uncertain, the system may ask for confirmation or offer to guide 
the user. 

While the above example illustrates an application where the multi-modal 
conversational computing system 10 is deployed in a vehicle, in another illustrative 
arrangement, the system can be deployed in a larger area, e.g., a room with multiple video 
input and speech input devices, as well as multiple associated devices controlled by the 
system 10. 

In accordance with the Strubbe/Geddes combination, the Office Action cites Geddes as 
disclosing an intent interpreter function. However, the intent interpreter function of Geddes is 
significantly different than the claimed concept of referential ambiguity resolution. 

Geddes is related to communication network management. The portions of Geddes cited in 
the Office Action (including column 7, lines 20-27; column 7, lines 35-65; column 8, lines 17-30) 
are therefore related to interpreting intent of a network operator. As clearly explained therein, 
Geddes uses "a simplified model of network operator intent consisting of three activities: {1) 
nominal network operations . . .(2) respond to security event . . . and (3) respond to network fault . 
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. . ." However, as column 6, line 64, through page 7, line 1, of Geddes state: "[t]he operator monitor 
205 monitors and records when a user presses a key, moves the mouse, or presses a mouse button 
. . . [t]he information collected can then be used ... to determine the present intention of the human 
operator." Thus, Geddes does not determine operator intent through anything other than the singular 
interface modality associated with operator keystrokes/mouse clicks via monitor 205. 

Thus, there is no disclosure in Geddes of "intent determination . . . based on at least a portion 
of the received multi-modal data," as in the claimed invention, since there is no multi-modal data 
in Geddes. Second, there is no disclosure in Geddes of "resolving referential ambiguity associated 
with the one or more users and the one or more devices in the environment based on at least a 
portion of the received multi-modal data," as in the claimed invention. With only one network 
operator and a typical keystroke/mouse click interface in Geddes, there is no need to "resolve 
referential ambiguity associated with one or more users and one or more devices in the 
environment." Geddes discussed no such ambiguity. Third, Strubbe is silent as to intent 
determination altogether. 

Accordingly, the Strubbe/Geddes combination fails to teach or suggest all of the limitations 
of the pending independent claims. 

Furthermore, as stated above, Geddes is related to communication network management. 
Strubbe is related to an entertainment device that simulates personal interaction and responds to a 
user's mental state and/or personality. Thus, the combination of Strubbe and Geddes is improper. 

The Federal Circuit has stated that when patentability turns on the question of obviousness, 
the obviousness determination "must be based on objective evidence of record" and that "this 
precedent has been reinforced in myriad decisions, and cannot be dispensed with." In re Lee , 277 
F.3d 1338, 1343 (Fed. Cir. 2002). Moreover, the Federal Circuit has stated that "conclusory 
statements" by an examiner fail to adequately address the factual question of motivation, which is 
material to patentability and cannot be resolved "on subjective belief and unknown authority." Id. 
at 1343-1344. 

In the Office Action at paragraph 4, the Examiner provides the following statement to prove 
motivation to combine Strubbe and Geddes, with emphasis supplied: "[i]t would have been obvious 
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to a person with ordinary skill in the art to determine the user's intent in the system Strubbe et al., 
because it would provide a convenient way using user cues to determine an object or topic to which 
a user is referring ." 

Applicants submit that this statement is based on the type of "subjective belief and unknown 
authority" that the Federal Circuit has indicated provides insufficient support for an obviousness 
rejection. More specifically, the Examiner fails to identify any objective evidence of record which 
supports the proposed combination. The cited references relate to disparate environments, disparate 
problems, and disparate solutions. There is no motivation given in the cited references to combine. 
Nor is there any support provided in the Office Action other than the subjective belief offered by the 
Examiner based on unknown authority. 

Applicants assert that all dependent claims are also patentably distinct over the 
Strubbe/Geddes combination for at least these reasons. 

Further, Adler fails to remedy the deficiencies pointed out above with respect to Strubbe and 
Geddes. Also, since Adler relates to a system and method for receiving, analyzing, and managing 
a database of calendar information obtained from a variety of source documents, Applicants assert 
that a combination of Strubbe, Geddes and Adler also violates the Federal Circuit's holding in the 
above-referenced In re Lee decision. Again, the Examiner's conclusory statement regarding 
"convenient data inputting" at paragraph 34 is not sufficient to support an obviousness rejection. 

In view of the above, Applicants believe that the pending claims are in condition for 
allowance, and respectfully request withdrawal of the§103(a) rejections. 



Respectfully submitted, 



Date: February 22, 2005 
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